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The purpose of this study is to adapt the computational thinking scale to 
Chinese. The study group consists of 1015 students. The study was 
performed in the descriptive scanning model. The final version of the 
scale was corrected in line with the opinions of the language experts who 
received the items translated from Turkish to Chinese. Exploratory and 
confirmatory factor analyses were calculated to determine the validity of 
the scale. Later, the distinctiveness forces were calculated. To determine 
the reliability of the scale internal consistency and stability levels were 
calculated. It has been concluded that the computational thinking scale is 
a valid and reliable tool in Chinese culture that can be used to determine 
high school students’ computational thinking skills. In addition, it was 
concluded that the students’ computational thinking skills were quite 
high. In terms of factors, the students! highest level skills are “Creativity” 
and the lowest ones are “Problem Solving” and “Algorithmic Thinking”. 
In terms of total scores and factors, computational thinking skills of male 
students are higher than female students. But problem solving skills are 
similar. It was concluded that k10 students' computational thinking skills 
were higher than k11 students in terms of “Problem Solving”, “Critical 
Thinking” and total scores. Based on the results obtained from this 
research and the literature, it is recommended that students frequently 
take part in activities that aim to improve their Problem Solving and 
Algorithmic Thinking skills, especially within the context of different 
courses. 


Introduction 


Computational Thinking (CT) as a concept has become popular in recent years. 
However, nowadays almost everyone, irrespective of age, is expected to have some basic 
computational thinking skills in parallel with the developments in technology. Hence, being a 
digital citizen requires students to possess CT skills. Computational thinking is a 
comprehensive thinking that includes mathematical thinking, engineering thinking and 
scientific thinking. It is an important carrier of innovative thinking ability. 
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Wing (2006) outlined the basic definition of CT as a way of “solving problems, designing 
systems and understanding human behaviors by drawing on the concepts of computer 
science”. The International Society for Technology in Education (ISTE) and Computer 
Science Teacher Association (CSTA) published an operational definition about computational 
thinking :CT is a way to use computer and other tools to solve the problem, involved in data 
collection, data analysis, data presentation, using the algorithm steps and resources for 
thinking to develop the optimal combination of automation solutions, and problem solving 
process can be extended to other areas (CSTA and ISTE,2011) .In 2016, the Computer 
Science Teacher Association (CSTA) updated their definition of computational thinking: 
computational thinking is a kind of the methodology to solve the problem, this method can be 
extended to all disciplines from the Computer Science field, providing a unique way for 
analyzing and developing problems that can be solved by the calculating method 
(CSTA,2016). It is possible to define Computational Thinking briefly as having the 
knowledge, skill and attitudes necessary to be able to use the computers in the solution of life 
problems for production purposes (Ozden, 2015). From the views of different scholars, we 
can see that while most scholars emphasize the characteristics of computational thinking as 
the ability to solve a problem, from different angles, the interpretation of computational 
thinking is different. 


Although there is no unified definition of computational thinking, its importance is obvious. 
When the fact that computational thinking has a border and general frame is taken into 
consideration, it is a valid basic skill not only for the computers, but also for everybody and it 
is considered that it will take place in the basic skills (reading, writing and arithmetic) used by 
everyone in the near future (Wing, 2006). Jeannette Wing presented CT and defined it as a 
skill for everyone, not just for computer scientists (Wing, 2006). Computational Thinking is a 
new perspective for children in k-12 to observe and understand the world around them. It's a 
new ability to understand and solve problems using computational processes and methods, 
and it's a necessary skill for them to deal with future competition and challenges. 


Nowadays, more researchers are paying attention to computational thinking. Especially the 
experts in the educational technology field have emphasized that Computational thinking is 
very significant in terms of the skills of the 21st century (Voogt, Fisser, Good, Mishra, & 
Yadav, 2015). And how to cultivate and evaluate students' computational thinking ability 
attracts their attention. There is no doubt that the most critical work in the development of 
computational thinking is evaluation. The evaluation of computational thinking plays an 
important role in the k-12 practice field, which is the basis of the CT training activities and 
the evidence of the training results. However, there is no widely accepted standard for 
evaluating CT in k-12. The existing research about evaluation standards of computational 
thinking in k-12 mostly adopted the multidimensional and hierarchical setting methods, 
combined with some content elements of science of computer and problem solving process, 
and using the grade as reference to divide different standards. Among them, the most 
authoritative one is the "Case guide" of computational thinking that different school age 
students should reach different levels from International Society for Technology in Education 
(ISTE) and Computer Science Teacher Association (CSTA).Computational thinking is 
divided into nine dimensions: data collection, data analysis, data presentation, problem 
decomposition, abstract, automation, simulation and parallel algorithm and process, and use 
case describes the reference performance behavior of computational thinking ability that k-12 
students in different stages need to master. 


Since computational thinking is the thinking process of solving problems, and problems are 
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generated in situations, most evaluation methods are carried out under certain circumstances. 
Currently, the most commonly used evaluation methods are text discourse analysis, subject 
test, work analysis, graphic analysis and behavior analysis. Discourse analysis based on text 
means to discover students’ thinking changes from their language, mainly through interviews 
and thinking-aloud. However, this kind of an interview is time-consuming and requires 
students to have a clear memory of the process to finish their work. For the students in the 
lower grade, there may be a situation of unclear expressions and irrelevant answers. 
Therefore, it is difficult and limited to use interview and other text discourse analysis in k-12. 
Testing refers to using some questions or problems to test students’ computational thinking at 
a certain stage of the course teaching, the results of the analysis can provide feedback to the 
teacher and students in a timely manner. At present, this way is used very often. It is very 
convenient, and can get students’ learning process information to a certain extent, but the 
design of the test is very critical, which requires certain theoretical or evaluation criteria (Koh 
et al.,2010; Aggarwal, Gardner-Mc Cune, Touretzky, 2017; Chen et al.,2017).Work analysis 
means analyzing student performances in various modules from their work and then 
diagnosing the improvement level of their ability in different aspects, but the operation is 
difficult, unless there is a specific analysis tool support(Roman —Gonzalez, Pérez —Gonzalez, 
Jiménez —Fernandez, 2016). Graphic analysis and behavior analysis mainly focus on students’ 
thinking process and behavior performance, which are the most difficult methods. Graphic 
analysis mainly adopts flowchart and pseudo-code to reflect students’ logical solution and 
thinking path when they solve problems. Behavior analysis begins from the students' learning 
behavior, analyzing students' problem solving in the practice process as well as error 
correction, the recycling process, analyzing the path to solve a problem, observing the 
application of various modules, evaluating students’ computational thinking performance. 
Field observation needs extra observers in the process of teaching activities to record the 
operating behavior of the students, and with the development of technology, the records of 
student behaviors can be obtained by using some recording screens. However, this method 
requires to record as many details as possible, which is time-consuming and requires a large 
amount of analysis (Esteves, Fonseca, Morgado, 2011). 


On the other hands, there is no scale found of which validity and reliability have been proven 
to measure the levels regarding especially the computational thinking skills in Chinese. 
Therefore, this research is limited with self-report based scale and other assessment 
techniques were not taken into consideration. The purpose of this study is to describe the 
students’ computational thinking skills and adapt the Computational Thinking Scales into 
Chinese from Turkish for determining the computational thinking skills of the high school 
students by filling this space in the literature. Hope this scale makes significance to measure 
cr 


Method 
Research Design 
In addition to being a scale adaptation study, this is a descriptive research. It is 


executed in the scanning model. In this context; students’ computational thinking skills have 
been tried to be determined. 
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Study Group 


The study group of this study consists of 1015 high school students at the levels of 
K10 and K11 degree in one private School in Ningxia Province and another public school in 
Jiangxi Province, China. While descriptive factor analysis was performed on data collected 
from K11 level students, confirmatory factor analysis was performed on data collected from 
K10 level students. All data were used in other analyses. The distribution of the students 
according to gender and class levels is summarized in Table 1. 


Table 1. The distribution of the study group according to the class and gender 


Sex 


Total 
Boy Girl 
K10 188 191 379 
Grade Kil 279 357 636 
Total 467 548 1015 


Measurement Tool 


The data of this study were collected using Computational Thinking Scale adapted to 
Chinese by researchers. The scale is designed for the first time by Korkmaz, Cakir and Ozden 
(2017) to measure computational thinking skills of university students in Turkey and its 
original name is “Computational Thinking scales (CTS)”. Later, this scale was adapted by 
Korkmaz, Cakir and Ozden (2015) to measure computational thinking skills of secondary 
school students in Turkey. Within the scope of this study, this scale was translated into 
Chinese and adapted to high school students’ in China. 


The scale designed by Korkmaz, Cakir and Ozden (2017) to measure the computational 
thinking skills of university students in Turkey consists of 29 items and five factors. The 
validity and reliability study of the scale was carried out separately in two different study 
groups consisting of students from faculty of education and faculty of engineering, and the 
other students in the science and literature faculty, theology faculty and health sciences 
faculty. Exploration factor analysis was performed in the first application and confirmatory 
factor analysis was performed in the second application. Parameters indicate acceptable 
compliance for this five-factor structure in both applications. The factors on the scale, the 
number of items and the internal consistency coefficients are summarized in Table 2. 


Table 2. Reliability analysis results considering the whole of the scale and its factors for 
undergraduate Students. 


Two congruent 


Factors Number halivas Spearman Guttmann Cronbach’s 
of items , Brown Split-Half Alpha 
correlation 
Creativity 8 713 832 832 843 
Algorithmic Thinking 6 .756 861 .860 869 
Cooperativity 4 835 .910 908 .865 
Critical Thinking 2) 562 719 .687 784 
Problem Solving 6 406 578 578 727 


Computational Thinking 29 


Skills 344 512 498 822 


The scale adapted by Korkmaz, Cakir and Ozden (2015) to measure the computational 
thinking skills of middle school students in Turkey consists of 22 items and five factors. 
Validity and reliability of the scale were carried out in the study group consisting of 241 
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students studying at K7 and K8 levels. A confirmatory factor analysis was performed on the 
obtained data and parameters indicate acceptable compliance for this five factor structure. The 
factors on the scale, the number of items and the internal consistency coefficients are 
summarized in Table 3. 


Table 3. Reliability analysis results considering the whole of the scale and its factors for 
middle school students. 


Factors Number of Cronbach’s Alpha 
items 
Creativity 4 .640 
Algorithmic Thinking 4 .762 
Cooperativity 4 811 
Critical Thinking 4 714 
Problem Solving 6 867 
Computational Thinking Skills 22 .809 


Scale Adaptation Process 


The first stage of the scale adaptation process is the translation phase (Hamleton & 
Patsula, 1999). At this stage, the original scale was translated from Turkish into English by a 
fluent Turkish and English-speaking educational technology expert and sent to a language 
specialist for examination. The translated scale was reviewed in case there are significant 
differences between expert’s translations. The translation process was completed by making 
necessary corrections. After this stage, the scale was translated from English into Chinese by 
a fluent Chinese and English speaking educational technology expert. The final translation 
form, as indicated by Hamleton and Patsula (1999), has been translated into English again by 
two experts and their consistency with the original material structures has been examined. In 
the examination, it was observed that the original scale materials and the texts in the form 
obtained by translation from Chinese were the language equivalence. 


The second stage of scale adaptation is the implementation of the adapted scale in the 
experimental group (Deniz, 2007). In this context, the adapted and corrected test should be 
applied to the Pilot Group before the examination of psychometric characteristics and should 
be checked for any further corrections to be made on the scale. In this way, after the draft 
scale form was created, the scale was applied to 1015 students at K10 and K11 level in order 
to evaluate the factor structure of the scale, structure validity and the reliability of the scale 
scores and the differences of the items. Factor structures for the Chinese form of the scale 
were studied based on the data obtained from the application. 


At the last stage, the data obtained after the draft scale was applied to the study group were 
uploaded to SPSS and Amos programs to perform validity and reliability analysis of the scale. 
The validity of the original factor structure of the scale in Chinese culture and high school 
level was investigated by both exploratory and confirmatory factor analysis (Giilbahar & 
Biytik6ztiirk, 2008). According to the basic parameters for exploratory and confirmatory 
factor analysis, it was concluded that the factor structure of the scale is valid in Chinese 
culture and K10 and K11 levels. Internal consistency analysis was performed on the data to 
calculate the reliability of the scale. 
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Data Analysis 


Each item has been scaled as never (1), rarely (2), occasionally (3), generally (4), 
always (5). The scores that are obtained from the answers given by students to five Likert 
type scale do not perform a standardized picture due to the differences of item numbers in 
factors. That is why it is appropriate to transform the obtained raw scores into standard scores 
the lowest of which is 20, the highest one is 100. That is because this developed scale aims to 
reach self-regulated learning score that can be standardized regardless of the features of the 
group it has been applied. The formula given below can be used in the transformation of raw 
scores into standard score: 


Xraw score 
Xstandard score x20 


Item numbers 
The levels that are the equivalents of scores obtained from sub scales can be given such: 20- 
51: Low Level;52-67: Medium Level; 68-100: High Level. On these data obtained in order to 
detect self-regulated learning levels of students; frequency, percentage, arithmetical means, 
standard deviation and t tests have been employed. In differentiation analyses p<0.05 
significance level has been considered sufficient. 


Finding 


Findings Regarding the Validity of the Scale 


The structural validity and item-factor score correlations were calculated and the results were 
presented below. 


Construct Validity 


Findings Regarding the Exploratory Factor Analysis: According to Tatlidil (2002), 
the data collected must first be tested for compliance with factor analysis. In this context, 
using Kaiser-Meyer-Oklin (KMO) and Bartlett tests, it was determined whether to perform 
factor analysis on these data. If the KMO value is between 0.70 and 0.80, the medium level, 
else if between 0.80 and 0.90, the good and over 0.90, the data set is considered to be 
perfectly suitable for factor analysis. In addition, if KMO is less than 0.50, the data set cannot 
be detected (Field, 2000; Russell, 2002). In this study, KMO= 0.858; Bartlett test value was 
x2= 6229.979; SD=231 (p=0.000). According to Bartlett test value, which is known as unit 
matrices, it is understood that the zero hypothesis was rejected at the level of 0.01 
significance (Biytik6ztiirk, 2002; Eroglu, 2008). In this context, it can be said that the data set 
is good for factor analysis. 


Based on the values obtained, exploratory and confirmatory factor analyses were used on the 
data; the condition of separation of the scale into the factors was determined by basic 
component analysis; and factor loads were investigated by using Varimax steep rotation 
technique. Factor analysis is used to determine whether items on a scale are divided into 
fewer factors (Balci1, 2009). The basic components analysis is a very common technique as a 
factorization technique (Biiytk6ztiirk, 2002). At the end of the basic components analysis 
used for factor analysis, the items in which there is at least 0.100 difference between the 
factor loads and whose factor load is less than 0.40 in other words, the items whose load is 
divided into the two factors need to be discarded (Biyiikéztiirk, 2002). Since the original 
scale consists of five factors, Varimax steep rotation technique was used according to the 
basic components without making the basic components analysis. Two items, one of which is 
less than 0.40, and the other of which is spread to different factors, were removed from the 
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scale. As a result of these operations, 20 items remaining in the scale can be collected under 
five factors similar to the original scale. It was determined that the KMO value of the 20-item 
scale was 0.883; the Bartlett values were y¥2=5475.929; SD=190; p<0.001. The basic criteria 
in evaluating factor analysis results are factor loads (Balci, 2009; Gorsuch, 1983; Eroglu, 
2008). The high factor load is seen as an indicator that the variable can be placed under the 
given factor (Biiyiik6éziitk, 2002). The factor loads of 20 items on the scale without being 
subjected to rotation (unrotated) were between 0.400 and 0.690; however, these loads were 
subjected to rotation after Varimax steep rotation technique, between .570 and .823. The 
explanation of at least 40% of the general variance in the literature is sufficient in terms of 
behavioral Sciences (Biiytk6ztirk, 2002; Eroglu, 2008; Klein, 1994; Scherer at al., 1988). It 
was determined that the items and factors included in the scale explained 56.312% of the total 
variance. When the contents of the factors in the next step were examined, it was observed 
that the original scale was preserved. In this context, factor names remained the same. In the 
slope accumulation graphic (Graph 1) drawn according to the values, the factor structure is 
also observed. In Graph 1, it means that there is a high acceleration decrease in the first five 
factors; therefore, there is an important contribution of these five factors to the variance; 
however, the decrease in other factors started to be horizontal; or, in other words, the 
contribution of the variance is close to each other (Biiyikotiirk, 2002; Eroglu, 2008). 
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Graph 1. Screen plot graphic (Eigenvalues according to the factors). 


As a result of these conducted processes, the findings regarding the item loads of the total 20 
items remaining in the scale according to the factors and the amounts of the factors in 
explaining the eigenvalues and variance are presented in Table 4. 


Table 4. Factor analysis results of the scale as per factors 


Items Com. Factor Fl F2 F3 F4 F5 
Ol .690 .823 
Cooperativity 2 TS B22 
03 .657 779 
04 .690 .656 
P3 566 731 
p4 573 .723 
Problem Solving P5 532 .666 
P2 427 658 
Pl 534 634 
T2 .690 .783 
Critical Thinking T3 .627 .738 
Tl 515 642 
T5 393 .602 
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A6 .647 .709 
Algorithmic A3 580 .708 
Thinking A4 .606 .698 
Al 467 .650 
CS 591 762 
Creativity C4 529 615 
Cl .400 570 
Eigenvalues 4.92 2.30 1.70 1.23 1.09 
Explained variance 13.05 12.46 11.96 11.11 Vad 


As shown in Table 4, The Cooperativity factor of the scale contains 4 items and the factor 
loads vary between 0.656 and 0.823. The eigenvalues of this factor is 4.92; the contribution it 
provides to the total variance is 13.05%. Problem solving factor contains 5 items. Factor loads 
of items are between 0.634 and 0.731. The eigenvalues of this factor is 2.30; the contribution 
it provides to the total variance is 12.46%. The Critical Thinking factor contains 4 items. 
Factor loads of items are between 0.602 and 0.783. The eigenvalues of this factor is 1.70; the 
contribution it provides to the total variance is 11.96%. The Algorithmic Thinking factor 
contains 4 items. Factor loads of items are between 0.650 and 0.709. The eigenvalues of this 
factor is 1.23; the contribution it provides to the total variance is 11.11%. The Creativity 
factor contains 3 substances. Factor loads of items are between 0.570 and 0.762. The 
eigenvalues of this factor is 1.09; the contribution it provides to the total variance is 7.72%. 


Findings Regarding the Confirmatory Factor Analysis: At the end of exploratory factor 
analysis, confirmatory factor analysis was carried out on data collected from K10 level 
students to verify the factor structure of the re-tested scale consisting of 5 factors. For each 
item, the estimate values are presented in Table 5 as a result of the confirmatory factor 
analysis using the maximum likelihoods technique without any limitation. 


Table 5. Standardized Regression Weights 
Creativity : 
Creativity 
Creativity 
Algorithmic Thinking 
Algorithmic Thinking 
Algorithmic Thinking 
Algorithmic Thinking 
Cooperativity 
Cooperativity 
Cooperativity 
Cooperativity 
Critical Thinking 
Critical Thinking 
Critical Thinking 
Critical Thinking 
Problem Solving 
Problem Solving 
Problem Solving 
Problem Solving 
Problem Solving 
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can be said that the estimate values are generally near to 0.70 and that there are no values that 
are far from 0.70. In confirmatory factor analysis, model-data compatibility is examined 
(Kline, 1994; Tabachnick & Fidell, 2001: Cited in Giilbahar & Biyiik6ztiirk, 2008). In the 
confirmatory factor analysis, a large number of adjustment indexes are used to assess the 
validity of the model. Among these, the most frequently used ones are Chi-Square, the 
goodness fit index (GFI), the corrected goodness fit index (AGFI), the square root of mean 
errors (RMR or RMS), and the mean square root of approximate errors (RMSEA) (Giilbahar 
& Biyiik6ziitiirk, 2008). In the literature, if the ratio of (y2/SD) calculated with DFA is less 
than 5, it can be seen as an indicator of the model's good compatibility with real data 
(MacCallum et al., 1996; Sumer, 2000). For model data compatibility, it is expected that GFI 
and AGFI values be higher than90, RMS or standardized RMS and RMSEA values be lower 
than ,05 (Sumer, 2000; Kline, 2005; Simsek, 2007). On the other hand the smallness of GFI 
value from 0.85, the highness of AGFI value from 0.80 and the smallness of RMS value from 
0.10 is taken as criteria indicating the compatibility of model with actual data (Anderson & 
Gerbing, 1984; Marsh et al., 1988; Siimer, 2000; Kline, 2005; Simsek, 2007). 


The confirmatory factor analysis is based on the principle that the correlations between the 
observed and the unobserved variables are evaluated and tested as a hypothesis (Pohlmann, 
2004). In this context, if the values obtained from confirmatory factor analysis are in the 
range of ¥2/D<3; 0<RMSEA<0.05; 0<s-RMR<0.05; 0.97<NFIX<1; 0.97<CFIX<1; 0.95<GFIs1; 
0.95<AGFIS1 and 0.95 < IFI < 1. they show perfect fit, and if they are in the range of 
¥2/D<5;  0.06<RMSEA<0.08; 0.06<S-RMR<0.08; 0.90<NFI<0.96; 0.90<CFI<0.96; 
0.90<GFI<0.96; 0.90<AG<0.96; 0.90<IFI<0.96 and 0.90 < IFI < 0.96 they show acceptable 
compatibility (Kline, 2005; Simsek, 2007). When the goodness of fit values obtained for CTS 
are examined, It was found as ¥2(sp=160, n=379)= 311.132 p<.001, CMIN/DF=1.945, RMSEA= 
0.050, S-RMR= 0.043, GFI= 0.92, AGFI= 0.90, CFI= 0.93 and IFI= 0.93. According to these 
values, it can be said that y2/D, RMSEA and S-RMR observed fit values were excellent; CFI, 
GFI, AGFI and IFI observed fit values were acceptable (Kline, 2005; Simsek, 2007). In other 
words, this model obtained shows that the factors are verified by means of the data. The 
values of factor model and factor-item relationship of the scale are given in Figure 1. 
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Figure 1. Confirmatory factor analysis diagram of the scale 
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Item Distinctiveness 


In this section, the correlations between the score attained from each item in the 
factors and the score attained from the factors have been calculated according to the item total 
and the distinctiveness levels of the items were tested. For each item, the item-factor 
correlation values are shown in Table 6. 


Tablo 6. Item-Factor Correlations 


F1. F2. F3 F4, F5. 
Creativity Alg. Thin. Cooperativity Crit. Thin. Prob. Sol. 
I r I. r I r I r I. r 


C1 .710¢**) Al .682(**) O1 — .809(***) T1 .752(¥*) Pl .654(**) 
C4 .726(**) A3 .728(**) 02 833(**) T2 817) P2 ~~ .676(**) 
C5 —.703(**) A4 .T7T9F*) O3  -.806(**) T3 .778(#*) P3 — .736(**) 

A6 .776(F*) O04 — .730(**) T5 643(#*) P4 — .737(**) 


P5~— .681(**) 
** 0<.001, N=1015 


As shown in Table 6, the correlation coefficiency of substance test was 0.703 to 0.726 for the 
first factor; 0.682 to 0.779 for the second factor; 0.730 to 0.833 for the third factor; 0.643 to 
0.817 for the fourth factor; 0.654 to 0.737 for the last factor. Each item has a meaningful and 
positive relationship with the overall of the factor (p<0.001). These coefficients are the 
validity coefficient of each substance and represent the overall consistency of the factor; in 
other words, the level of service to the overall purpose of the factor (Carminini & Zeller, 
1982). In this context, it can be said that the level of distinctiveness of each substance is quite 
high. 


Findings Regarding the Reliability of the Scale 


Internal consistency and stability analyses were performed on the data to calculate the 
reliability of the scale. The reliability analysis of the scale was calculated by using Cronbach 
Alpha reliability coefficient. The reliability analysis of each factor and the overall scale is 
summarized in Table 7: 


Table 7. Reliability analysis results considering the whole of the scale and its factors. 


Factors Item Numbers eeonbace 
Alpha 
Creativity (C) 3 .613 
Algorithmic Thinking (A) 4 727 
Cooperativity (O) 4 .805 
Critical Thinking (T) 4 .739 
Problem Solving (P) 5 .734 
Computational Thinking Levels 20 830 


As shown in Table 7, Cronbach Alpha reliability coefficient of the scale consisting of 5 sub- 
factors and 20 items was 0.830. On the other hand, Cronbach alpha values for factors are 
between 0.613 and 0.805. Although Cronbach alpha for Creativity factor is below 0.70, 
Cronbach alpha values for other factors and CTS are above 0.70, indicating that the internal 
consistency of the scale is high enough. 


The stability level of the scale was determined using the test-retest method. The final form of 
the scale was applied to 36 students two weeks after the application. The relationship between 


the scores obtained at the end of both applications was examined both in terms of each item 
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and in terms of the overall scale. In this way, the ability to make stable measurements of both 


the items and the overall of the scale has been tested and the results are summarized in Table 
8. 


Table 8. Test-retest results 


F1. F2. F3 F4. FS. 

Creativity Alg. Think. Cooperativity Crit. Thin. Prob. Sol. 

I. r I. r I. r I. r I. r 

Cl 824(#*) Al 612(**) Ol — .822(**) T1 = .763(**) = PI .803(**) 

C4 .668(**) AZ .769(**) O2 — .805(**"*) T2  .877(**) = P2-—.930(**) 

C5 652(**) A4 — .859(**) O3 — .805(**) T3 .905(**) =P3  .981(**) 
A6 — .730(**) O04 — 427(**) T5 .898(**) = P4 — .982(**) 

P5 —_.908(**) 
Fl .784(#*) F2 884(#*) F3. .894(**) F493 6(**) FS .978(**) 


Total .923(**) 
N: 36; **=p<0.001 


Correlation coefficients obtained by test-retest method of each item change between 0.427 
and 0.982 and they are observed that each relationship is meaningful and positive in Table 8. 
Correlation coefficients obtained by test-retest method of the factors forming the scale range 
between .784 and .978. The correlation between total scores is .923 and it is observed that 
each relationship is meaningful and positive. Accordingly, it can be said that the scale can 
make stable measurements. 


Findings on Students’ Computational Thinking Skills 
Students’ computational thinking skills levels are summarized in Table 9. 


Table 9. Students’ Computational Thinking Levels 


os : Level (f/%) 

Factors N X = Sd. Min Max raw Medium High 
Creativity (C) 77.3 11.1 33 100 4 A 249 24.5 762 75.1 
Algorithmic Thinking (A) 67.5 13.8 20 100 124 12.2 404 39.8 487 48.0 
Cooperativity (O) 75.2 146 20 100 54. 5.3. 237 23.3 724 71.3 
Critical Thinking (T) 1015 73.6 13.3 20 100 49 48 280 27.6 686 67.6 
Problem Solving (P) 67.1 13.3 20 100 151 14.9 306 30.1 558 55.0 
Computational Thinking 71.6 8.8 31 100 i 1.1 311 30.6 693 683 
Levels 


As shown in Table 9, students’ computational thinking skills scores range from 31 to 100; 
means is X=71.6. It is seen on the table that, 68.3% of the students’ skill levels are high, 
30.6% of the students’ skill levels are medium and 1.1% of the students’ skill levels are low. 
According to this, it can be said that students have very high computational thinking skills. 


When we look at the scores of the factors one by one, it is observed that the factor with the 
highest mean is “Creativity” (X=77.3) and the factors with the lowest mean is “Algorithmic 
Thinking” (X=67.1) and “problem solving” (X=67.1). On the other hand, the highest level of 
skill in the high Group is “Creativity” (75.1%), the lowest level of the high Group is 
“algorithmic Thinking” (48%). According to this, it can be said that the highest level of 
students! skills in terms of factors is “Creativity”, the lowest ones are “problem solving” and 
“Algorithmic Thinking". Table 10 summarizes the findings regarding the students ' level of 
computational thinking skills by gender. 
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Table 10. The Effect of Gender on Students' Computational Thinking Skills 


Factors N X Sd t df p 

Creativity (C) — ae be: oe 5.29 1013  .000 
Algorithmic Thinking (A) ae ue ays yg 7.33 1013 .000 
Cooperativity (O) aaa ie i 2.78 1013 .006 
Critical Thinking (T) ae ai i Iy5 613 1013 .000 
Problem Solving (P) em se oe Py 1.03 1013300 
Computational Thinking Levels pass oo 1. 2 6.48 1013 .000 


Female 548 69.9 7.8 


As shown in Table 10, there is a significant difference between the students' computational 
thinking skills by gender (t(2-1013)=6.48; p<0.001). When the averages are examined, it is 
observed that differentiation is in favor of males. When the factors were examined, there was 
a significant difference in favor of males in all other factors except the problem solving factor 
(t(2-1013=1.03; p<0.05). It can be said that computational thinking skills of males are higher 
than those of females, but they are similar in terms of “Problem Solving” skills. Table 11 
summarizes the findings related to computational thinking skills levels according to the 
classroom level of the students. 


Table 11. Effect of Class Level on Students' Computational Thinking Skills 


Factors N X Sd t df p 

Creativity (C) oe ut ae -.31 1013 .760 
Algorithmic Thinking (A) ae a i .13 1013 .896 
Cooperativity (O) ae oe i es 1.24 1013 .213 
Critical Thinking (T) a ae = 1.83 1013 .050 
Problem Solving (P) aa oe ae oe 3.17 1013 .002 
Cepia Wik E38 TA 98s iota 


As Show in Table 11, There is a significant difference between the total scores of 
computational thinking according to the students’ class levels (t(2-1013)=2.147; p<0.05) . When 
the factors were examined, there was a significant difference between the Critical Thinking 
(t(2-1013)=1.83; p<0.05) and the Problem Solving skills (t(2-1013;=3.17; p<0.05), and there was 
no difference between the other factors. When the averages are examined, it is observed that 
differentiation is in favor of students at K10 level. According to this, students with K10 level 
of Problem Solving, Critical Thinking and total scores can be said to be higher than students 
with K11 level. 


Conclusion and Discussion 


In this study, the “Computational Thinking Scale” was adapted to Chinese in order to 
determine Chinese students’ computational thinking skill levels. The scale is a five-digit likert 
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type scale, consisting of 20 items that can be collected under five factors. Exploratory and 
confirmatory factor analysis was performed to verify the factor structure of the scale. The 
factor analysis is based on the analysis of the factors, the factor load, the factors’ eigenvalues, 
and the explanatory variance ratios of the factors and it can be said that the scale is a scale 
with structural validity. A confirmatory factor analysis was carried out to verify the factor 
structures of the scale, which was determined to be composed of 5 factors as a result of 
exploratory factor analysis. According to the results of confirmatory factor analysis, the 
observed values of the scale model were determined to be acceptable for the CFI, GFI, AGFI 
and IFI indices, and to be perfect for the y2/D, RMSEA and S-RMR. 


The correlation between the score obtained from each item and the score obtained from the 
factor to which the item belongs is used as a criterion in terms of understanding the level of 
the item to serve the overall purpose of the factor (Balci, 2009). In this respect, the correlation 
between each item of the scale and the points obtained from the factor to which the item 
belongs varies between 0.643 and 0.833. It can be said that each item in the scale and each 
factor serves a significant purpose to measure the quality of the scale in general and each item 
is distinguished at the desired level. Internal consistency coefficients of the scale were 
calculated using the Cronbach Alpha formula. Cronbach Alpha reliability coefficient of the 
scale was determined as .830. Stability level of the scale test- retest method and obtained the 
correlation coefficients were determined to be between .784 -.978 and the correlation in the 
range of the total score was .923. It can be said that the scale can perform reliable 
measurements within the framework of these values. As a matter of fact, the reliability 
coefficient of 0.70 and above is considered to be an indicator of the reliability of the scale 
(Biytik6ziitk, 2002; Gorsuch, 1983). As a result, it can be said that “Computational thinking 
Scale” is a valid and reliable scale that can be used to determine computational thinking skill 
levels of high school students at K10 and K11 levels in China. 


On the other hand, the following results have been obtained regarding the students' 
computational thinking skills: 


Students’ computational thinking skills are quite high. In terms of factors, the students’ 
highest level skills are “Creativity’ and the lowest ones are “Problem Solving” and 
“Algorithmic Thinking’. Similar results were obtained in the scope of the research conducted 
by Korkmaz and his colleagues (2015) to examine the computational thinking skills of 
university students in terms of different variables. In this research, it is emphasized that 
students perceive their own computational thinking skills at a moderate and high level. In 
addition, it is emphasized that the students’ Problem Solving skills are low and cooperation 
skills are high then the other skills. In both studies, it was concluded that students’ problem- 
Solving and algorithmic thinking skills were lower than other skills. If the students are facing 
some obstacles while trying to gain access to a specific purpose or understanding, there is a 
problem for that person (Aksoy, 2004). Learning to overcome the problems students may face 
in the future is one of the primary goals of the schools. The process of solving the problem 
should be combined and the problem should be used in the solution (Soylu & Soylu, 2006). 
Algorithmic thinking is defined as the ability to understand, implement, evaluate and produce 
algorithms (Brown, 2015). In summary, algorithmic thinking and problem-solving skills are 
important skills among the 21st century’s skills. It can be said that the students have one of 
these skills, which means that they can have the other skills. Therefore, it can be said that it is 
natural for these two skills to be involved in the lowest or highest group together. 


In terms of total scores and factors, computational thinking kills of males are higher than 
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females’, but similar for Problem Solving. Research conducted by Korkmaz and his 
colleagues (2015) shows that gender is effective on critical thinking skills, and that males feel 
more confident about critical thinking skills than females. In the study conducted by Korkmaz 
(2009), although there is no difference in the tendency and levels of critical thinking by 
gender, males are more curious and more confident than female students. In addition, it is 
possible to come up with the results that males feel more comfortable about using computer 
technologies than females. 


Problem solving, Critical thinking and total scores of K10 level students in terms of 
computational thinking skills are higher than K11 level students. In the research conducted by 
Korkmaz and his colleagues (2015), similarly, the computational thinking skills of graduate 
students decrease as the class level progresses. Accordingly, it can be said that schools have 
not been able to contribute to computational thinking skills during the education process. 


As a result, if computational thinking skills are summarized as a kind of problem-solving 
approach (ISTE, 2015; Wing, 2006; Barr, Harrison & Conery, 2011) that strengthens human 
thinking skills with technology, it can be said that it is important for individuals to acquire 
and develop these skills within their educational processes. In particular, it is stated that when 
we examine the above-mentioned factors (creative thinking, algorithmic thinking, critical 
thinking, problem solving and cooperation skills), it is stated that individuals will have these 
skills in school age, develop themselves and have a digital age learning culture (ISTE, 2015; 
Barr et al., 2011; Brown, 2015; Aksoy, 2004; Giintic, Odabas1 & Kuzu, 2013; Grover & Pea, 
2013; Lye & Koh, 2014). Barr and his colleagues (2011) emphasize that students should 
acquire these skills in school age so that they can transfer these skills to other problem 
situations. Brown (2015) emphasized that developing these skills in schools would be an 
important gain in considering that everyday life is surrounded by algorithms and complex 
problems. Based on the results achieved by this research and the literature, it is recommended 
that students frequently take part in activities that aim to improve their problem solving and 
algorithmic thinking skills, especially in the context of different courses. 
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Appendinx 1: CTS 


Fact. Items 
Cl I like the people who are sure of most of their decisions FRE ARE A CHR EAR A fa DA 
C4 I have a belief that I can solve the problems possible to occur | F¥4A{8 4 ERTE— 7 HHO PBA EI, «Fe aT 
when I encounter with a new situation DI APE EELS [a] Bi, 
CS I trust my intuitions and feelings of “trueness” and “wrongness” | EAR Dea] EMO, Fetafa A CEMA” Fei 
hen I h the solution of bl os SEs. 
when I approach the solution of a problem HOTEL REMIT. 
Al I can immediately establish the equity that will give the solution | #¥23 v7 BIAS FEA Gy CHE ARR [a] 
of a problem 
A3 I think that I learn better the instructions made with the help of | Fei), ARR TE IR LECLERC, Fe 
mathematical symbols and concepts AH BINA BH, 
A4 I believe that I can easily catch the relation between the figures FRU NFR AT LUBA EE FZ [AIR 
A6 I can digitize a mathematical problem expressed verbally. Fea LE 4 iB SS RIA BCS | eS (ES 
HAY (PN, TTAB) 
Ol I like experiencing cooperative learning together with my group | F¥= Willa] 2— EET AES 
friends. 
02 In the cooperative learning, I think that I attain/will attain more | F¥i\ HFRABEETE ATES S PARE Aa, KK 
successful results because I am working in a group. RAV) A RR 2°25 19, 
03 [like solving problems related to group project together with my | 7F-A(/E{ SF, Feet WAI-AA — eR E28 
friends in cooperative learning. wi eH fa) 
om FRAY Bo 
04 More ideas occur in cooperative learning. BUNCE 5 Bota A — HET GES SOE 
, RAAT EHS MAE, 
Tl I am good at preparing regular plans regarding the solution of | RH Kill APR & Ze [a] BAT RI 
the complex problems. 
T2 It is fun to try to solve the complex problems. Bei Aik ik 2S LAY [ER RY 
T3 lam willing to learn challenging things. Fe eS AOA PK ERPERO FRG 
T5S I make use of a systematic method while comparing the options | 7 /FHVEPEAIR EIR, EXT RAW SE. 
at my hand and while reaching a decision. 
Pl I have problems in the demonstration of the solution of a | fA pve Agve a) MIN ARH, WHR 
problem in my mind. ALY 
P2 I have problems in the issue of where and how I should use the | 7FfZ e/a] MINOR A, Fe EOE EM LEA ani 
variables such as X and Y in the solution of a problem. a Ai a =A 
48 FX ALY pa Be, 
P3 I cannot apply the solution ways I plan respectively and HEAR GEG MARAIS MISA RA 
gradually. aatat 7 
P4 I cannot produce so many options while thinking of the possible | 7-43 pEARUE— 7 [a] AQ TAM, FRA REALE 
solution ways regarding a problem. Bo Fie SK 
P5 I cannot develop my own ideas in the environment of | 7F- A/F SANE, FercveeAL a CAA 


cooperative learning. 
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